Tangibility visualization of virtual objects within a computer-generated reality environment

ABSTRACT

The present disclosure relates to techniques for providing tangibility visualization of virtual objects within a computer-generated reality (CGR) environment, such as a CGR environment based on virtual reality and/or a CGR environment based on mixed reality. A visual feedback indicating tangibility is provided for a virtual object within a CGR environment that does not correspond to a real, tangible object in the real environment. A visual feedback indicating tangibility is not provided for a virtual representation of a real object within a CGR environment that corresponds to a real, tangible object in the real environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/375,595, entitled “TANGIBILITY VISUALIZATION OF VIRTUAL OBJECTSWITHIN A COMPUTER-GENERATED REALITY ENVIRONMENT”, filed Apr. 4, 2019,which claims priority to U.S. Provisional Patent Application Ser. No.62/664,759, entitled “TANGIBILITY VISUALIZATION OF VIRTUAL OBJECTSWITHIN A VIRTUAL ENVIRONMENT,” filed Apr. 30, 2018, the contents ofwhich are hereby incorporated by reference in their entirety.

FIELD

This application relates generally to a computer-generated realityenvironment, and more specifically to techniques for providingtangibility visualization of virtual objects within a computer-generatedreality environment.

BACKGROUND

Computers can completely project or partially superimposecomputer-generated images on a user's view to provide acomputer-generated reality environment that can be experienced by theuser. A computer-generated reality environment can be based on differenttypes of realities. An electronic device optionally detects the user'sreal movements and projects and simulates those movements within aseries of visual images or video of the computer-generated realityenvironment. Through these movements projected or simulated within thecomputer-generated reality environment, the user can interact withobjects within the computer-generated reality environment.

BRIEF SUMMARY

The present disclosure describes techniques for providing tangibilityvisualization of virtual objects within a computer-generated reality(CGR) environment, where the CGR environment provides a user interactingwith the CGR environment with a realistic and immersive experience.Because the experience is realistic and immersive, the user can easilyconfuse a virtual (and thus intangible) object within the CGRenvironment as being a real tangible object that exists outside of theCGR environment.

Thus, the described techniques enhance user convenience and furtherprovide the user with an enhanced degree of safety when interacting witha CGR environment by enabling the user to quickly and easily visuallyrecognize whether an object within the CGR environment is a non-tangiblevirtual object or corresponds to a real, and thus tangible, object inthe real environment.

Additionally, the described techniques are not limited to providingtangible visualization to a particular type of CGR environment, butrather can be implemented in any type of CGR environment. Theseenvironments include, for example, CGR environments based on mixedreality and CGR environments based on virtual reality.

In accordance with some embodiments, a method is described. The methodcomprises: presenting a computer-generated reality environmentcomprising a representation of a first real object and a displayedvirtual object; in accordance with detecting, using one or more sensors,a movement of the first real object, presenting a corresponding movementof the representation of the first real object from a first location toa second location in the computer-generated reality environment; and inaccordance with a determination that a distance criteria is satisfiedand the virtual object does not correspond to a second real objectdetected using the one or more sensors, removing a portion of thedisplayed virtual object, wherein the distance criteria comprises athreshold distance between the second location and the virtual object.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium storing one or more programs configured to be executed byone or more processors of an electronic device, the one or more programsincluding instructions for: presenting a computer-generated realityenvironment comprising a representation of a first real object and adisplayed virtual object; in accordance with detecting, using one ormore sensors, a movement of the first real object, presenting acorresponding movement of the representation of the first real objectfrom a first location to a second location in the computer-generatedreality environment; and in accordance with a determination that adistance criteria is satisfied and the virtual object does notcorrespond to a second real object detected using the one or moresensors, removing a portion of the displayed virtual object, wherein thedistance criteria comprises a threshold distance between the secondlocation and the virtual object.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium storing one or more programs configured to be executed by one ormore processors of an electronic device, the one or more programsincluding instructions for: presenting a computer-generated realityenvironment comprising a representation of a first real object and adisplayed virtual object; in accordance with detecting, using one ormore sensors, a movement of the first real object, presenting acorresponding movement of the representation of the first real objectfrom a first location to a second location in the computer-generatedreality environment; and in accordance with a determination that adistance criteria is satisfied and the virtual object does notcorrespond to a second real object detected using the one or moresensors, removing a portion of the displayed virtual object, wherein thedistance criteria comprises a threshold distance between the secondlocation and the virtual object.

In accordance with some embodiments, an electronic device is described.The electronic device comprises one or more processors and memorystoring one or more programs configured to be executed by the one ormore processors, the one or more programs including instructions for:presenting a computer-generated reality environment comprising arepresentation of a first real object and a displayed virtual object; inaccordance with detecting, using one or more sensors, a movement of thefirst real object, presenting a corresponding movement of therepresentation of the first real object from a first location to asecond location in the computer-generated reality environment; and inaccordance with a determination that a distance criteria is satisfiedand the virtual object does not correspond to a second real objectdetected using the one or more sensors, removing a portion of thedisplayed virtual object, wherein the distance criteria comprises athreshold distance between the second location and the virtual object.

In accordance with some embodiments, an electronic device is described.The electronic device comprises: means for presenting acomputer-generated reality environment comprising a representation of afirst real object and a displayed virtual object; means, in accordancewith detecting, using one or more sensors, a movement of the first realobject, for presenting a corresponding movement of the representation ofthe first real object from a first location to a second location in thecomputer-generated reality environment; and means, in accordance with adetermination that a distance criteria is satisfied and the virtualobject does not correspond to a second real object detected using theone or more sensors, for removing a portion of the displayed virtualobject, wherein the distance criteria comprises a threshold distancebetween the second location and the virtual object.

DESCRIPTION OF THE FIGURES

FIGS. 1A-1B depict exemplary systems for use in various CGRtechnologies, including virtual reality and mixed reality.

FIGS. 2A-2D illustrate an exemplary technique for providing visualfeedback indicating tangibility in a CGR environment.

FIGS. 3A-3D illustrate an exemplary technique for providing visualfeedback indicating tangibility in a CGR environment.

FIGS. 4A-4B are a flow diagram illustrating a method for providingvisual feedback indicating tangibility within a CGR environment, inaccordance with some embodiments.

The embodiments depicted in the figures are only exemplary. One skilledin the art will readily recognize from the following discussion thatalternative embodiments of the structures and methods illustrated hereincan be employed without departing from the principles described herein.

DETAILED DESCRIPTION

Various examples of electronic systems and techniques for using suchsystems in relation to various computer-generated reality technologiesare described.

A physical environment (or real environment) refers to a physical worldthat people can sense and/or interact with without aid of electronicsystems. Physical environments, such as a physical park, includephysical articles (or physical objects or real objects), such asphysical trees, physical buildings, and physical people. People candirectly sense and/or interact with the physical environment, such asthrough sight, touch, hearing, taste, and smell.

In contrast, a computer-generated reality (CGR) environment refers to awholly or partially simulated environment that people sense and/orinteract with via an electronic system. In CGR, a subset of a person'sphysical motions, or representations thereof, are tracked, and, inresponse, one or more characteristics of one or more virtual objectssimulated in the CGR environment are adjusted in a manner that comportswith at least one law of physics. For example, a CGR system may detect aperson's head turning and, in response, adjust graphical content and anacoustic field presented to the person in a manner similar to how suchviews and sounds would change in a physical environment. In somesituations (e.g., for accessibility reasons), adjustments tocharacteristic(s) of virtual object(s) in a CGR environment may be madein response to representations of physical motions (e.g., vocalcommands).

A person may sense and/or interact with a CGR object using any one oftheir senses, including sight, sound, touch, taste, and smell. Forexample, a person may sense and/or interact with audio objects thatcreate a 3D or spatial audio environment that provides the perception ofpoint audio sources in 3D space. In another example, audio objects mayenable audio transparency, which selectively incorporates ambient soundsfrom the physical environment with or without computer-generated audio.In some CGR environments, a person may sense and/or interact only withaudio objects.

Examples of CGR include virtual reality and mixed reality.

A virtual reality (VR) environment (or virtual environment) refers to asimulated environment that is designed to be based entirely oncomputer-generated sensory inputs for one or more senses. A VRenvironment comprises a plurality of virtual objects with which a personmay sense and/or interact. For example, computer-generated imagery oftrees, buildings, and avatars representing people are examples ofvirtual objects. A person may sense and/or interact with virtual objectsin the VR environment through a simulation of the person's presencewithin the computer-generated environment, and/or through a simulationof a subset of the person's physical movements within thecomputer-generated environment.

In contrast to a VR environment, which is designed to be based entirelyon computer-generated sensory inputs, a mixed reality (MR) environmentrefers to a simulated environment that is designed to incorporatesensory inputs from the physical environment, or a representationthereof, in addition to including computer-generated sensory inputs(e.g., virtual objects). On a virtuality continuum, a mixed realityenvironment is anywhere between, but not including, a wholly physicalenvironment at one end and virtual reality environment at the other end.

In some MR environments, computer-generated sensory inputs may respondto changes in sensory inputs from the physical environment. Also, someelectronic systems for presenting an MR environment may track locationand/or orientation with respect to the physical environment to enablevirtual objects to interact with real objects (that is, physicalarticles from the physical environment or representations thereof). Forexample, a system may account for movements so that a virtual treeappears stationary with respect to the physical ground.

Examples of mixed realities include augmented reality and augmentedvirtuality.

An augmented reality (AR) environment refers to a simulated environmentin which one or more virtual objects are superimposed over a physicalenvironment, or a representation thereof. For example, an electronicsystem for presenting an AR environment may have a transparent ortranslucent display through which a person may directly view thephysical environment. The system may be configured to present virtualobjects on the transparent or translucent display, so that a person,using the system, perceives the virtual objects superimposed over thephysical environment. Alternatively, a system may have an opaque displayand one or more imaging sensors that capture images or video of thephysical environment, which are representations of the physicalenvironment. The system composites the images or video with virtualobjects, and presents the composition on the opaque display. A person,using the system, indirectly views the physical environment by way ofthe images or video of the physical environment, and perceives thevirtual objects superimposed over the physical environment. As usedherein, a video of the physical environment shown on an opaque displayis called “pass-through video,” meaning a system uses one or more imagesensor(s) to capture images of the physical environment, and uses thoseimages in presenting the AR environment on the opaque display. Furtheralternatively, a system may have a projection system that projectsvirtual objects into the physical environment, for example, as ahologram or on a physical surface, so that a person, using the system,perceives the virtual objects superimposed over the physicalenvironment.

An augmented reality environment also refers to a simulated environmentin which a representation of a physical environment is transformed bycomputer-generated sensory information. For example, in providingpass-through video, a system may transform one or more sensor images toimpose a select perspective (e.g., viewpoint) different than theperspective captured by the imaging sensors. As another example, arepresentation of a physical environment may be transformed bygraphically modifying (e.g., enlarging) portions thereof, such that themodified portion may be representative but not photorealistic versionsof the originally captured images. As a further example, arepresentation of a physical environment may be transformed bygraphically eliminating or obfuscating portions thereof.

An augmented virtuality (AV) environment refers to a simulatedenvironment in which a virtual or computer generated environmentincorporates one or more sensory inputs from the physical environment.The sensory inputs may be representations of one or more characteristicsof the physical environment. For example, an AV park may have virtualtrees and virtual buildings, but people with faces photorealisticallyreproduced from images taken of physical people. As another example, avirtual object may adopt a shape or color of a physical article imagedby one or more imaging sensors. As a further example, a virtual objectmay adopt shadows consistent with the position of the sun in thephysical environment.

There are many different types of electronic systems that enable aperson to sense and/or interact with various CGR environments. Examplesinclude head mounted systems, projection-based systems, heads-updisplays (HUDs), vehicle windshields having integrated displaycapability, windows having integrated display capability, displaysformed as lenses designed to be placed on a person's eyes (e.g., similarto contact lenses), headphones/earphones, speaker arrays, input systems(e.g., wearable or handheld controllers with or without hapticfeedback), smartphones, tablets, and desktop/laptop computers. A headmounted system may have one or more speaker(s) and an integrated opaquedisplay. Alternatively, a head mounted system may be configured toaccept an external opaque display (e.g., a smartphone). The head mountedsystem may incorporate one or more imaging sensors to capture images orvideo of the physical environment, and/or one or more microphones tocapture audio of the physical environment. Rather than an opaquedisplay, a head mounted system may have a transparent or translucentdisplay. The transparent or translucent display may have a mediumthrough which light representative of images is directed to a person'seyes. The display may utilize digital light projection, OLEDs, LEDs,uLEDs, liquid crystal on silicon, laser scanning light source, or anycombination of these technologies. The medium may be an opticalwaveguide, a hologram medium, an optical combiner, an optical reflector,or any combination thereof. In one example, the transparent ortranslucent display may be configured to become opaque selectively.Projection-based systems may employ retinal projection technology thatprojects graphical images onto a person's retina. Projection systemsalso may be configured to project virtual objects into the physicalenvironment, for example, as a hologram or on a physical surface.

FIG. 1A and FIG. 1B depict exemplary system 100 for use in variouscomputer-generated reality technologies.

In some examples, as illustrated in FIG. 1A, system 100 includes device100 a. Device 100 a includes various components, such as processor(s)102, RF circuitry(ies) 104, memory(ies) 106, image sensor(s) 108,orientation sensor(s) 110, microphone(s) 112, location sensor(s) 116,speaker(s) 118, display(s) 120, and touch-sensitive surface(s) 122.These components optionally communicate over communication bus(es) 150of device 100 a.

In some examples, elements of system 100 are implemented in a basestation device (e.g., a computing device, such as a remote server,mobile device, or laptop) and other elements of the system 100 areimplemented in a head-mounted display (HMD) device designed to be wornby the user, where the HMD device is in communication with the basestation device. In some examples, device 100 a is implemented in a basestation device or a HMD device.

As illustrated in FIG. 1B, in some examples, system 100 includes two (ormore) devices in communication, such as through a wired connection or awireless connection. First device 100 b (e.g., a base station device)includes processor(s) 102, RF circuitry(ies) 104, and memory(ies) 106.These components optionally communicate over communication bus(es) 150of device 100 b. Second device 100 c (e.g., a head-mounted device)includes various components, such as processor(s) 102, RF circuitry(ies)104, memory(ies) 106, image sensor(s) 108, orientation sensor(s) 110,microphone(s) 112, location sensor(s) 116, speaker(s) 118, display(s)120, and touch-sensitive surface(s) 122. These components optionallycommunicate over communication bus(es) 150 of device 100 c.

In some examples, system 100 is a mobile device. In some examples,system 100 is a head-mounted display (HMD) device. In some examples,system 100 is a wearable HUD device.

System 100 includes processor(s) 102 and memory(ies) 106. Processor(s)102 include one or more general processors, one or more graphicsprocessors, and/or one or more digital signal processors. In someexamples, memory(ies) 106 are one or more non-transitorycomputer-readable storage mediums (e.g., flash memory, random accessmemory) that store computer-readable instructions configured to beexecuted by processor(s) 102 to perform the techniques described below.

System 100 includes RF circuitry(ies) 104. RF circuitry(ies) 104optionally include circuitry for communicating with electronic devices,networks, such as the Internet, intranets, and/or a wireless network,such as cellular networks and wireless local area networks (LANs). RFcircuitry(ies) 104 optionally includes circuitry for communicating usingnear-field communication and/or short-range communication, such asBluetooth®.

System 100 includes display(s) 120. In some examples, display(s) 120include a first display (e.g., a left eye display panel) and a seconddisplay (e.g., a right eye display panel), each display for displayingimages to a respective eye of the user. Corresponding images aresimultaneously displayed on the first display and the second display.Optionally, the corresponding images include the same virtual objectsand/or representations of the same physical objects from differentviewpoints, resulting in a parallax effect that provides a user with theillusion of depth of the objects on the displays. In some examples,display(s) 120 include a single display. Corresponding images aresimultaneously displayed on a first area and a second area of the singledisplay for each eye of the user. Optionally, the corresponding imagesinclude the same virtual objects and/or representations of the samephysical objects from different viewpoints, resulting in a parallaxeffect that provides a user with the illusion of depth of the objects onthe single display.

In some examples, system 100 includes touch-sensitive surface(s) 122 forreceiving user inputs, such as tap inputs and swipe inputs. In someexamples, display(s) 120 and touch-sensitive surface(s) 122 formtouch-sensitive display(s).

System 100 includes image sensor(s) 108. Image sensors(s) 108 optionallyinclude one or more visible light image sensor, such as charged coupleddevice (CCD) sensors, and/or complementary metal-oxide-semiconductor(CMOS) sensors operable to obtain images of physical objects from thereal environment. Image sensor(s) also optionally include one or moreinfrared (IR) sensor(s), such as a passive IR sensor or an active IRsensor, for detecting infrared light from the real environment. Forexample, an active IR sensor includes an IR emitter, such as an IR dotemitter, for emitting infrared light into the real environment. Imagesensor(s) 108 also optionally include one or more event camera(s)configured to capture movement of physical objects in the realenvironment. Image sensor(s) 108 also optionally include one or moredepth sensor(s) configured to detect the distance of physical objectsfrom system 100. In some examples, system 100 uses CCD sensors, eventcameras, and depth sensors in combination to detect the physicalenvironment around system 100. In some examples, image sensor(s) 108include a first image sensor and a second image sensor. The first imagesensor and the second image sensor are optionally configured to captureimages of physical objects in the real environment from two distinctperspectives. In some examples, system 100 uses image sensor(s) 108 toreceive user inputs, such as hand gestures. In some examples, system 100uses image sensor(s) 108 to detect the position and orientation ofsystem 100 and/or display(s) 120 in the real environment. For example,system 100 uses image sensor(s) 108 to track the position andorientation of display(s) 120 relative to one or more fixed objects inthe real environment.

In some examples, system 100 includes microphones(s) 112. System 100uses microphone(s) 112 to detect sound from the user and/or the realenvironment of the user. In some examples, microphone(s) 112 includes anarray of microphones (including a plurality of microphones) thatoptionally operate in tandem, such as to identify ambient noise or tolocate the source of sound in space of the real environment.

System 100 includes orientation sensor(s) 110 for detecting orientationand/or movement of system 100 and/or display(s) 120. For example, system100 uses orientation sensor(s) 110 to track changes in the positionand/or orientation of system 100 and/or display(s) 120, such as withrespect to physical objects in the real environment. Orientationsensor(s) 110 optionally include one or more gyroscopes and/or one ormore accelerometers.

FIGS. 2A-2D illustrate an exemplary technique for providing visualfeedback indicating tangibility in a CGR environment that includes onlyvirtual objects according to an embodiment of the present disclosure.

In FIG. 2A, a CGR environment 200 (e.g., based on virtual reality)includes a plurality of virtual objects 202-210 (e.g., virtual object202 of a chair, virtual object 204 of a keyboard, virtual object 206 ofa monitor, virtual object 208 of a desk, virtual object 210 of a lamp).

In FIG. 2B, device 100 a presents (e.g., displays, causes display of, orallows viewing of), on display 120, a representation 212 (e.g., acomputer-simulated reality representation or a direct, see-through view)in CGR environment 200 corresponding to a user-controlled real object inthe real environment (e.g., a body portion of the user of device 100 a,such as a hand, face, or head of the user; an object that is held orworn by the user in the real environment or otherwise controlled by theuser in the real environment) that is controlled by the user of device100 a. Representation 212 simulates, within CGR environment 200,movements of the corresponding real object within the real environment.For example, if the user-controlled real object is the hand of the user,representation 212 simulates, within CGR environment 200, movements ofthe user's hand in the real environment. In some embodiments, theuser-controlled real object (e.g., the user's hand) corresponding torepresentation 212 is detected via one or more sensors (e.g., imagesensors) of device 100 a, for instance, when the user-controlled realobject is in the field of vision of the one or more sensors of device100 a. In some embodiments, the user-controlled real object (e.g., theuser's hand) is additionally or alternatively detected via one or moresensors (e.g., image sensors, motion sensors, accelerometers,gyroscopes) external to device 100 a (e.g., sensors installed in thereal environment, such as a room, whereupon CGR environment 200 isbased).

In some embodiments, display 120 of device 100 a is a transparentdisplay that enables a direct view of real objects in the realenvironment through the transparent display, and device 100 a presentsCGR environment 200 on the transparent display. In some embodiments, ifdisplay 120 is a transparent display, representation 212 correspondingto the user-controlled real object (e.g., the user's hand) is a directview of the user-controlled real object (e.g., a direct view of theuser's hand) in the real environment through the transparent display.

In some embodiments, if representation 212 corresponds to a hand of theuser of the device, device 100 a concurrently displays (or causesdisplay of), on display 120, a second representation corresponding tothe other hand of the user of the device (e.g., when both hands aredetected within the field of vision of device 100 a) in CGR environment200. In some embodiments, device 100 a displays (or causes display of),on display 120, a different representation corresponding to a differentbody portion of the user (e.g., the feet of the user) in CGR environment200.

In some embodiments, if representation 212 corresponds to a hand of theuser of the device, device 100 a displays representation 212 within CGRenvironment 200 as a generic hand shape (e.g., a generic blue hand thatis easily visible within the CGR environment). In some embodiments,representation 212 visually resembles the corresponding hand of the userin the real environment (e.g., similar size, similar skin tone, similaraccessories, such as a ring).

In FIG. 2B, the user-controlled real object (e.g., the user's hand thatis projected within CGR environment 200 as representation 212) is atleast a threshold distance (e.g., 6 inches, 1 foot) away from any ofvirtual objects 202-210 within CGR environment 200. In some embodiments,device 100 a detects (e.g., via one or more internal and/or externalimage sensors) the user-controlled real object (e.g., the user's hand)moving from its position in FIG. 2B to a position within CGR environment200 that is less than a threshold distance away from a virtual object(e.g., virtual object 202) within CGR environment 200. In someembodiments, the threshold distance is predefined (e.g., by an operatingsystem of device 100 a or set by a user of device 100 a).

In FIG. 2C, upon detecting (e.g., via one or more internal and/orexternal image sensors) that the user-controlled real object (e.g., theuser's hand that is projected within CGR environment 200 asrepresentation 212) is within the threshold distanced (e.g., 6 inches, 1foot) from virtual object 202 within CGR environment 200, device 100 adisplays (or causes display of), on display 120, a visual feedback 214(e.g., a graphical effect, such as an aperture effect) that visuallyindicates to the user that virtual object 202 is a non-tangible virtualobject and that the user-controlled real object (e.g., the user's hand)is within the threshold distance form the non-tangible virtual object.

In FIG. 2C, the user-controlled real object (e.g., user's hand) iswithin the threshold distance from virtual object 202 but at least thethreshold distance away from other virtual objects 204-210 of CGRenvironment 200. In some embodiments, the threshold distance dynamicallychanges depending upon the speed of the user-controlled real object's(e.g., the user's hand's) movement in the real environment. For example,if device 100 a detects (e.g., via one or more internal and/or externalimage sensors) that the speed of the movement of the user-controlledreal object (e.g., the user's hand) is greater than a threshold speed,device 100 a increases the threshold distance from virtual object 202 atwhich display of visual feedback 214 is triggered. For another example,if device 100 a detects (e.g., via one or more internal and/or externalimage sensors) that the speed of the movement of the user-controlledreal object (e.g., the user's hand) is less than a threshold speed,device 100 a decreases the threshold distance from virtual object 202 atwhich display of visual feedback 214 is triggered. In some embodiments,the threshold distance dynamically changes depending upon changes in thespeed (e.g., acceleration or deceleration) of the user-controlled realobject's (e.g., the user's hand) movement in the real environment. Forexample, if device 100 a detects (e.g., via one or more internal and/orexternal image sensors) an acceleration in the speed of the movement ofthe user's hand, device 100 a increases the threshold distance fromvirtual object 202 at which display of visual feedback 214 is triggered.For another example, if device 100 a detects (e.g., via one or moreinternal and/or external image sensors) a deceleration in the speed ofthe movement of the user's hand, device 100 a decreases the thresholddistance from virtual object 202 at which display of visual feedback 214is triggered.

In some embodiments, visual feedback 214 is provided via a graphicaleffect that is an aperture effect, where at least a portion of virtualobject 202 (e.g., an area of the virtual object surrounding the user'shand and/or proximate to the user's hand within the virtual realityenvironment) corresponding to the visual feedback that is within apredetermined distance from representation 212 ceases to be displayed(e.g., faded out, grayed out, blurred out) such that the portion ofvirtual object 202 is no longer visible to the user on the display. Insome embodiments, concurrently with the fading out effect, a region thatwas previously behind the faded-out portion of virtual object 202 in CGRenvironment 200 (e.g., because it was hidden from view by virtual object202) becomes visible to the user on display 120. In some embodiments,the “fade-out effect is displayed on display 120 with a cloud-likepattern.

In some embodiments, visual feedback 214 is a dynamic aperture effectwhere the magnitude of the effect changes in accordance with therelative distance of the user-controlled real object (e.g., user's hand)to virtual object 202 within CGR environment 200. In some embodiments,visual feedback 214 (e.g., gradually) increases in magnitude as thedistance between the user-controlled real object (e.g., the user's hand)and virtual object 202 decreases until the user-controlled real objectreaches an area occupied by (e.g., corresponds to, overlaps with)virtual object 202 within CGR environment 200. In some embodiments,visual feedback 214 (e.g., gradually) decreases in magnitude as thedistance between the user-controlled real object (e.g., the user's hand)and virtual object 202 increases until the user-controlled real objectis more than the threshold distance (e.g., 6 inches, 1 foot) away fromvirtual object 202 (e.g., at which point visual feedback 214 is nolonger displayed). In some embodiments, visual feedback 214 has apredetermined maximum size (e.g., at which point the dynamic apertureeffect no longer increases in magnitude), where the predeterminedmaximum size is reached once the user-controlled real object (e.g., theuser's hand that is projected within CGR environment 200 as SRrepresentation 212) reaches a region of CGR environment 200 that isoccupied by (e.g., corresponds to, overlaps with) virtual object 202and/or passes through virtual object 202 within CGR environment 200.

In some embodiments, the rate of change of the magnitude of a visualfeedback corresponds to (e.g., is proportional to) the rate of change ofthe speed of the movement of the user-controlled real object (e.g., theuser's hand). In some embodiments, as the user-controlled real object(e.g., the user's hand) approaches virtual object 202 at an increasinglyfaster speed, corresponding visual feedback 214 increases in magnitudeat an (e.g., proportionally) increasing rate. In some embodiments, asthe user-controlled real object (e.g., the user's hand) approachesvirtual object 202 at an increasingly slower speed, corresponding visualfeedback 214 increases in magnitude at a (e.g., proportionally)decreasing rate. In some embodiments, as the distance between theuser-controlled real object (e.g., the user's hand) and virtual object202 increases at an increasingly faster speed, corresponding visualfeedback 214 decreases in magnitude at an (e.g., proportionally)increasing rate. In some embodiments, as the distance between theuser-controlled real object (e.g., the user's hand) and virtual object202 increases at an increasingly slower speed, corresponding visualfeedback 214 decreases in magnitude at a (e.g., proportionally)decreasing rate.

In FIG. 2D, the user-controlled real object (e.g., the user's hand thatis projected within CGR environment 200 as representation 212) is passedthrough virtual object 202 within CGR environment 200 and is within thethreshold distance (e.g., 6 inches, 1 foot) from virtual object 204(e.g., a virtual keyboard) and virtual object 208 (e.g., a virtualdesk). In some embodiments, upon detecting (e.g., via one or moreinternal and/or external image sensors) that the user-controlled realobject (e.g., the user's hand) has “passed through” virtual object 202,device 100 a maintains visual feedback 214 corresponding to virtualobject 202 at a constant magnitude.

In some embodiments, further upon detecting (e.g., via one or moreinternal and/or external image sensors) that the user-controlled realobject (e.g., the user's hand) is within the threshold distance (e.g., 6inches, 1 foot) from virtual object 204 (e.g., a virtual keyboard),device 100 a displays (or causes display of), on display 120, a visualfeedback 216 similar to visual feedback 214, where visual feedback 216is an aperture effect on virtual object 204. In some embodiments,further upon detecting (e.g., via one or more internal and/or externalimage sensors) that the user-controlled real object (e.g., the user'shand) is within the threshold distance (e.g., 6 inches, 1 foot) fromvirtual object 208 (e.g., a virtual desk), device 100 a displays (orcauses display of), on display 120, a visual feedback 218 similar tovisual feedbacks 214 and 216, where visual feedback 218 is an apertureeffect on virtual object 208.

In FIG. 2D, visual feedback 214 corresponding to virtual object 202(e.g., a virtual chair) is larger in magnitude than both visual feedback216 corresponding to virtual object 204 (e.g., a virtual keyboard) andvirtual object 208 (e.g., a virtual desk) because the user-controlledreal object (e.g., the user's hand that is projected within CGRenvironment 200 as representation 212) is passed through virtual object202 but has not yet reached (e.g., though is within the thresholddistance of) virtual objects 204 and 208. Further, visual feedback 216corresponding to virtual object 204 is larger in magnitude than visualfeedback 218 corresponding to virtual object 204 because theuser-controlled real object (e.g., the user's hand) is closer to virtualobject 204 than to virtual object 208 within CGR environment 200.

FIGS. 3A-3D illustrate an exemplary technique for providing visualfeedback indicating tangibility in a CGR environment (e.g., based onmixed reality) that includes both virtual and real objects. In someembodiments, the CGR environment is projected onto a real environment(e.g., an enclosed area, such as a room) using one or more sensors(e.g., image sensors) that capture the contours of the real environment,and the projected CGR environment is displayed to the user via display120 of device 100 a.

As shown in FIG. 3A, a CGR environment 300 (e.g., based on mixedreality) includes a plurality of virtual objects 302-306 (e.g., avirtual object 302 of a chair, a virtual object 304 of a keyboard, avirtual object 306 of a monitor) and a plurality of real or virtualrepresentations 308-310 of real objects in the real environment (e.g., avirtual representation 308 of a real desk in the real environment, avirtual representation 310 of a real lamp in the real environment).Unlike the plurality of real or virtual representations 308-310, theplurality of virtual objects 302-306 are virtual (and thus non-tangible)and do not correspond to any real objects outside of CGR environment300.

In FIG. 3B, device 100 a displays (or causes display of), on display120, a representation 312 (e.g., a computer-simulated realityrepresentation or a direct, see-through view) in CGR environment 300corresponding to a user-controlled real object in the real environment(e.g., a body portion of the user of device 100 a, such as a hand, face,or head of the user; an object that is held or worn by the user in thereal environment or otherwise controlled by the user in the realenvironment) that is controlled by the user of device 100 a (e.g., sameas or similar to representation 212 described above with reference toFIGS. 2A-2D). In some embodiments, the user-controlled real object(e.g., the user's hand) corresponding to representation 312 is detectedvia one or more sensors (e.g., image sensors) of device 100 a, forinstance, when the user-controlled real object is in the field of visionof the one or more sensors of device 100 a. In some embodiments, theuser-controlled real object (e.g., the user's hand) is additionally oralternatively detected via one or more sensors (e.g., image sensors,motion sensors, accelerometers, gyroscopes) external to device 100 a(e.g., sensors installed in the real environment, such as a room,whereupon CGR environment 300 is based).

In some embodiments, display 120 of device 100 a is a transparentdisplay that enables a direct view of real objects in the realenvironment through the transparent display, and device 100 a presentsCGR environment 300 on the transparent display. In some embodiments, ifdisplay 120 is a transparent display, representation 312 correspondingto the user-controlled real object (e.g., the user's hand) is a directview of the real object in the real environment through the transparentdisplay.

In some embodiments, if representation 312 corresponds to a hand of theuser of the device, device 100 a concurrently displays (or causesdisplay of), on display 120, a second representation corresponding tothe other hand of the user of the device (e.g., when both hands aredetected within the field of vision of device 100 a) in CGR environment300. In some embodiments, device 100 a displays (or causes display of),on display 120, a different representation corresponding to a differentbody portion of the user (e.g., the feet of the user) in CGR environment300.

In some embodiments, if representation 312 corresponds to a hand of theuser of the device, device 100 a displays representation 3212 within CGRenvironment 300 as a generic hand shape (e.g., a generic blue hand thatis easily visible within the CGR environment). In some embodiments,representation 312 visually resembles the corresponding hand of the userin the real environment (e.g., similar size, similar skin tone, similaraccessories, such as a ring).

In FIG. 3B, representation 312 is at least (or more than) a thresholddistance (e.g., 6 inches, 1 foot) away from any of virtual objects302-306 and virtual representations 308-310 of mixed reality environment300. In some embodiments, while representation 312 is at least (or morethan) the threshold distance away from any of virtual objects 302-306and virtual representations 308-310, device 100 a detects (e.g., via oneor more internal and/or external image sensors) that the user-controlledreal object (e.g., the user's hand) is moving from its position in FIG.3B to a position that is less than the threshold distance away from oneor more virtual objects and/or virtual representations within CGRenvironment 300.

In FIG. 3C, upon detecting (e.g., via one or more internal and/orexternal image sensors) that the user-controlled real object (e.g.,user's hand) is within the threshold distance (e.g., 1 foot) fromvirtual object 302 (e.g., a virtual chair) within CGR environment 300,device 100 a displays (or causes display of), on display 120, a visualfeedback 314 (e.g., a graphical effect, such as an aperture effect,similar to visual feedback 214 described with reference to FIGS. 2A-2D).In FIG. 3C, the user-controlled real object (e.g., the user's hand) iswithin the threshold distance from virtual object 302 but not within thethreshold distance from virtual objects 304-306 and virtualrepresentations 308-310 of mixed reality environment 300.

In FIG. 3D, the user-controlled real object (e.g., the user's hand) ispassed through virtual object 302 and is within the threshold distance(e.g., 6 inches, 1 foot) from virtual object 304 (e.g., a virtualkeyboard) and virtual representation 308 (e.g., corresponding to a realdesk in the real environment). In some embodiments, upon detecting(e.g., via one or more internal and/or external image sensors) that theuser-controlled real object (e.g., the user's hand) is passed throughvirtual object 302, device 100 a maintains visual feedback 314corresponding to virtual 302 at a constant magnitude.

In some embodiments, upon detecting (e.g., via one or more internaland/or external image sensors) that the user-controlled real object(e.g., the user's hand) is within the threshold distance from virtualobject 304 (e.g., a virtual keyboard), device 100 a displays (or causesdisplay of), on display 120, a visual feedback 316 on virtual object 304similar to visual feedback 314 on virtual object 302 (e.g., an apertureeffect over at least a portion of virtual object 304 such that a portionof the CGR environment behind virtual object 304 that was hidden fromview is now visible through the aperture effect).

In some embodiments, upon detecting (e.g., via one or more internaland/or external image sensors) that the user-controlled real object(e.g., the user's hand) is within the threshold distance (e.g., 6inches, 1 foot) from virtual representation 308 (e.g., a virtualrepresentation corresponding to a real desk in the real environment),device 100 a does not display (or cause display of) a visual feedbackbecause virtual representation 308 corresponds to a real object in thereal environment (unlike virtual objects 302 and 304, which are virtualobjects (and thus non-tangible) that do not correspond to any realobjects in the real environment). Thus, in FIG. 3D, even though theuser-controlled real object (e.g., the user's hand) is passed throughvirtual object 302 and within the threshold distance (e.g., 6 inches, 1foot) from virtual object 304 and virtual representation 308, a visualfeedback is only generated and displayed by device 100 a for virtualobjects 302 and 304, and not for virtual representation 308.

By providing visual feedback (e.g., visual feedback 314 and 316) forvirtual objects but not for virtual representations 308-310 of realobjects, device 100 a enables the user to quickly and easily recognizewhether particular objects within mixed reality environment 300 arenon-tangible virtual objects or are virtual representations of real andtangible objects, and to quickly and easily distinguish between thenon-tangible virtual objects and virtual representations of real andtangible objects within the surrounding environment (e.g., therebyenhancing user safety by minimizing confusion on the part of the user asto which items are virtual, and thus not tangible, and which items arereal, and thus tangible).

In some embodiments, the visual feedback (e.g., visual feedback 214 onvirtual object 202, visual feedback 314 on virtual object 302) describedabove with reference to FIGS. 2A-2D and FIGS. 3A-3D are generated forvirtual objects of a CGR environment based on augmented reality. In someembodiments, the CGR environment based on augmented reality is displayedto a user on a display of device 100 a, where the display is (at leastpartially) a transparent display that enables the user to directly viewthe surrounding real environment through the display and virtual objectsdisplayed over the real environment to augment the real environment. Insome embodiments, the visual feedback described above with reference toFIGS. 2A-2D and FIGS. 3A-3D are provided for virtual objects within theCGR environment based on augmented reality, thereby quickly and moreclearly indicating to the user that the virtual objects are in factvirtual.

FIGS. 4A-4B are a flow diagram illustrating an exemplary processperformed by an electronic device (e.g., device 100 a). In someembodiments, the device has a display. In some embodiments, the displayis a (at least partially) transparent display. In some embodiments, theelectronic device is connected to and in communication with a displaythat is separate from the device. In some embodiments, the electronicdevice has one or more sensor devices (e.g., image sensor(s) 108,orientation sensor(s) 110, location sensor(s) 116). In some embodiments,the electronic device is connected to and in communication with one ormore sensor devices (e.g., image sensor(s) 108, orientation sensor(s)110, location sensor(s) 116) that are separate from the device. In someembodiments, the electronic device is a head-mounted device. In someembodiments, the electronic device is separate from but is secured on(or configured to be secured to) a head-mounted device. In someembodiments, the electronic device includes one or more speakers (e.g.,speaker(s) 118) for outputting audio. In some embodiments, theelectronic device is connected (or configured to be connected) to (e.g.,via wireless connection, via wired connection) and in communication (orconfigured to be in communication) with one or more speakers (e.g.,speaker(s) 118) for outputting audio.

At block 402, the electronic device (e.g., 100 a) presents (e.g., allowsviewing of, displays, or causes display of, on display 120) a CGRenvironment (e.g., 200, 300) comprising a (e.g., real or virtual)representation of a first real object (e.g., a user or body part of auser, such as the hand of the user; 212, 312) and a displayed virtualobject (e.g., 202, 204, 206, 208, 210, 302, 304, 306, 308, 310). In someembodiments, at block 404, the CGR environment is based on virtualreality (e.g., CGR environment 200). In some embodiments, at block 406,the CGR environment is based on mixed reality (e.g., CGR environment300), including augmented reality.

In some embodiments, the representation of the first real object (e.g.,212, 312) corresponds to a presented image of the first real object inthe CGR environment. In some embodiments, the representation of thefirst real object (e.g., 212, 312) corresponds to a direct view of thefirst real object through a display presenting the CGR environment.

At block 408, in accordance with detecting, using one or more sensors, amovement of a first real object (e.g., a body part of the user), theelectronic device (e.g., 100 a) presents (e.g., allows viewing of,displays, or causes display of, on display 120) a corresponding movementof the (e.g., real or virtual) representation of the first real object(e.g., 212, 312) from a first location to a second location in the CGRenvironment. In some embodiments, the first real object is a body partof the user (e.g., a hand of the user, a face or head of the user, afoot of the user).

At block 410, in accordance with a determination that a distancecriteria is satisfied and the virtual object (e.g., 202, 204, 206, 208,210, 302, 304, 306, 310) does not correspond to a second real objectdetected using the one or more sensors, the electronic device (e.g., 100a) removes a portion (e.g., 214, 216, 218, 314, 316) of the displayedvirtual object, where the distance criteria comprises a thresholddistance between the second location and the virtual object. In someembodiments, the electronic device (e.g., 100 a) determines whether thevirtual object corresponds to the second real object based onreconstruction data extracted from an image of the real environmentcaptured using the one or more sensors. In some embodiments, the virtualobject does not correspond to the second real object when the virtualobject is not a representation of the second real object in the CGRenvironment.

In some embodiments, the portion (e.g., 214, 216, 218, 314, 316) of thedisplayed virtual object (e.g., 202, 204, 208, 210, 302, 304, 306, 310)is removed via an aperture effect.

In some embodiments, at block 412, in accordance with a determinationthat the distance criteria is satisfied and the virtual object (e.g.,308) corresponds to the second real object, the electronic device (e.g.,100 a) forgoes removing the portion of the displayed virtual object(e.g., 202, 204, 208, 210, 302, 304, 306, 310). In some embodiments(e.g., prior to presenting a CGR environment comprising a virtual objectand a virtual representation of a user), the electronic device (e.g.,100 a) obtains an image of a third real object using the one or moresensors and presents, in the CGR environment (e.g., 300), arepresentation of the third real object (e.g., 308). In some examples,the electronic device (e.g., 100 a) forgoes removing the portion of thedisplayed virtual object in accordance with a determination that thedistance criteria is satisfied and the virtual object (e.g., 308)corresponds to the second real object detected using the one or moresensors. In some embodiments, the one or more sensors that are used arethe same as the one or more sensors that are used to detect the movementof the first real object at block 410. In some embodiments, the one ormore sensors that are used are different from the one or more sensorsthat are used to detect the movement of the first real object at block410.

In some embodiments, at block 414, in response to removing the portionof the displayed virtual object, the electronic device (e.g., 100 a)presents, in the removed portion, a second virtual object different fromthe virtual object (e.g., 202, 204, 208, 210, 302, 304, 306, 310).

In some embodiments, in accordance with a determination that a speed ofthe representation of the first real object (e.g., 212, 312) from thefirst location to the second location in the CGR environment correspondsto a first speed that is greater than a baseline speed, the thresholddistance is greater than a baseline threshold distance. In someembodiments, in accordance with a determination that the speed of therepresentation of the first real object (e.g., 212, 312) from the firstlocation to the second location in the CGR environment corresponds to asecond speed that is less than the baseline speed, the thresholddistance is less than the baseline threshold distance.

In some embodiments, in accordance with a determination that a speed ofthe representation of the first real object (e.g., 212, 312) isincreasing from the first location to the second location in the CGRenvironment, the threshold distance is greater than a baseline thresholddistance. In some embodiments, in accordance with a determination thatthe speed of the representation of the first real object (e.g., 212,312) is decreasing from the first location to the second location in theCGR environment, the threshold distance is less than the baselinethreshold distance.

In some embodiments, the threshold distance between the second locationand the virtual object comprises the distance between the representationof the first real object (e.g., 212, 312) in the CGR environment and thedisplayed virtual object in the CGR environment.

In some embodiments, a magnitude of the portion of the displayed virtualobject (e.g., the area of the displayed virtual object) that is removedis determined based on a distance between the second location in the CGRenvironment and the displayed virtual object in the CGR environment.

In some embodiments, at block 416, in accordance with detecting, usingthe one or more sensors, a second movement of the first real object(e.g., the hand of the user) from the second location to a thirdlocation in the CGR environment, the electronic device (e.g., 100 a)presents, at block 418, a corresponding movement of the representationof the first real object (e.g., 212, 312) from the second location tothe third location in the CGR environment (e.g., 200, 300). In someembodiments, at block 416, in accordance with detecting, using the oneor more sensors, a second movement of the first real object (e.g., thehand of the user) from the second location to a third location in theCGR environment, in accordance with a determination that a seconddistance criteria is satisfied and the virtual object (e.g., 202, 204,206, 208, 210, 302, 304, 306, 310) does not correspond to the secondreal object, the electronic device (e.g., 100 a) removes, at block 420,a second portion of the displayed virtual object, where the secondportion is different from the portion of the displayed virtual objectthat was removed. In some examples, the amount of the second portionremoved is based on a distance between the representation of the firstreal object at the third location and the virtual object. In someexamples, the second distance criteria comprises a determination thatthe representation of the first real object at the third location iswithin a second threshold distance of the virtual object. In someexamples, the value of the second threshold distance is based on a speedor acceleration of the second movement.

In some embodiments, at block 416, in accordance with detecting, usingthe one or more sensors, a second movement of the first real object(e.g., the hand of the user) from the second location to a thirdlocation in the CGR environment, in accordance with a determination thata second distance criteria is satisfied and the virtual object (e.g.,202, 204, 206, 208, 210, 302, 304, 306, 310) does not correspond to thesecond real object, the electronic device (e.g., 100 a) removes, atblock 422, an additional portion of the displayed virtual object fromthe displayed virtual object with the portion removed, where the seconddistance criteria is met when the third location is within a secondthreshold distance of the virtual object, the second threshold distancebeing less than the threshold distance. In some embodiments, theelectronic device (e.g., 100 a) continues to remove additional portionsof the displayed virtual object in response to detecting that therepresentation of the real object (e.g., 212, 312) is movingincreasingly closer to the virtual object in the CGR environment.

In some embodiments, at block 416, in accordance with detecting, usingthe one or more sensors, a second movement of the first real object(e.g., the hand of the user) from the second location to a thirdlocation in the CGR environment, in accordance with a determination thatthe distance criteria is no longer satisfied, the electronic device(e.g., 100 a) re-presents (e.g., re-displays, on display 120), at block424, the removed portion of the displayed virtual object (e.g., 202,204, 206, 208, 210, 302, 304, 306, 310).

The foregoing descriptions of specific embodiments and processes, asdescribed with reference to FIGS. 2A-2D, 3A-3D, and FIGS. 4A-4Brespectively, have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit thescope of the claims to the precise forms disclosed, and it should beunderstood that many modifications and variations are possible in lightof the above descriptions.

What is claimed is:
 1. An electronic device, comprising: one or moreprocessors; and memory storing one or more programs configured to beexecuted by the one or more processors, the one or more programsincluding instructions for: presenting a computer-generated realityenvironment comprising a representation of a first real object and adisplayed virtual object; detecting, using one or more sensors, amovement of the first real object; presenting a corresponding movementof the representation of the first real object from a first location toa second location in the computer-generated reality environment; and inaccordance with a determination that a distance criteria is satisfiedand the virtual object does not correspond to a second real objectdetected using the one or more sensors, removing a portion of thedisplayed virtual object, wherein the distance criteria comprises athreshold distance between the second location and the virtual object.2. The electronic device of claim 1, the one or more programs furtherincluding instructions for: in accordance with a determination that thedistance criteria is satisfied and the virtual object corresponds to thesecond real object, forgoing removing the portion of the displayedvirtual object.
 3. The electronic device of claim 1, wherein the virtualobject does not correspond to the second real object when the virtualobject is not a representation of the second real object in thecomputer-generated reality environment.
 4. The electronic device ofclaim 1, the one or more programs further including instructions for:determining whether the virtual object corresponds to the second realobject based on reconstruction data extracted from an image of a realenvironment captured using the one or more sensors.
 5. The electronicdevice of claim 1, the one or more programs further includinginstructions for: in response to removing the portion of the displayedvirtual object, presenting, in the removed portion, a second virtualobject different from the virtual object.
 6. The electronic device ofclaim 1, wherein the portion of the displayed virtual object is removedvia an aperture effect.
 7. The electronic device of claim 1, wherein therepresentation of the first real object corresponds to a presented imageof the first real object in the computer-generated reality environment.8. The electronic device of claim 1, wherein the representation of thefirst real object corresponds to a direct view of the first real objectthrough a display presenting the computer-generated reality environment.9. The electronic device of claim 1, wherein the first real object is ahand of the user.
 10. The electronic device of claim 1, the one or moreprograms further including instructions for: obtaining an image of athird real object using the one or more sensors; and presenting, in thecomputer-generated reality environment, a representation of the thirdreal object.
 11. The electronic device of claim 1, wherein: inaccordance with a determination that a speed of the representation ofthe first real object from the first location to the second location inthe computer-generated reality environment corresponds to a first speedthat is greater than a baseline speed, the threshold distance is greaterthan a baseline threshold distance; and in accordance with adetermination that the speed of the representation of the first realobject from the first location to the second location in thecomputer-generated reality environment corresponds to a second speedthat is less than the baseline speed, the threshold distance is lessthan the baseline threshold distance.
 12. The electronic device of claim1, wherein: in accordance with a determination that a speed of therepresentation of the first real object is increasing from the firstlocation to the second location in the computer-generated realityenvironment, the threshold distance is greater than a baseline thresholddistance; and in accordance with a determination that the speed of therepresentation of the first real object is decreasing from the firstlocation to the second location in the computer-generated realityenvironment, the threshold distance is less than the baseline thresholddistance.
 13. The electronic device of claim 1, wherein the thresholddistance between the second location and the virtual object comprisesthe distance between the representation of the first real object in thecomputer-generated reality environment and the displayed virtual objectin the computer-generated reality environment.
 14. The electronic deviceof claim 1, wherein a magnitude of the portion of the displayed virtualobject that is removed is determined based on a distance between thesecond location in the computer-generated reality environment and thedisplayed virtual object in the computer-generated reality environment.15. The electronic device of claim 1, the one or more programs furtherincluding instructions for: in accordance with detecting, using the oneor more sensors, a second movement of the first real object from thesecond location to a third location in the computer-generated realityenvironment: presenting a corresponding movement of the representationof the first real object from the second location to the third locationin the computer-generated reality environment; and in accordance with adetermination that the distance criteria is satisfied and the virtualobject does not correspond to the second real object, removing a secondportion of the displayed virtual object, the second portion differentfrom the portion of the displayed virtual object that is removed. 16.The electronic device of claim 15, the one or more programs furtherincluding instructions for: further in accordance with detecting, usingthe one or more sensors, a second movement of the first real object: inaccordance with a determination that a second distance criteria issatisfied and the virtual object does not correspond to the second realobject, removing an additional portion of the displayed virtual objectfrom the displayed virtual object with the portion removed, wherein thesecond distance criteria is satisfied when the third location is withina second threshold distance of the virtual object, the second thresholddistance smaller than the threshold distance.
 17. The electronic deviceof claim 15, the one or more programs further including instructionsfor: further in accordance with detecting, using the one or moresensors, a second movement of the first real object: in accordance witha determination that the distance criteria is no longer satisfied,representing the removed portion of the displayed virtual object.
 18. Anon-transitory computer-readable storage medium storing one or moreprograms configured to be executed by one or more processors of anelectronic device, the one or more programs including instructions for:presenting a computer-generated reality environment comprising arepresentation of a first real object and a displayed virtual object;detecting, using one or more sensors, a movement of the first realobject; presenting a corresponding movement of the representation of thefirst real object from a first location to a second location in thecomputer-generated reality environment; and in accordance with adetermination that a distance criteria is satisfied and the virtualobject does not correspond to a second real object detected using theone or more sensors, removing a portion of the displayed virtual object,wherein the distance criteria comprises a threshold distance between thesecond location and the virtual object.
 19. The non-transitorycomputer-readable storage medium of claim 18, the one or more programsfurther including instructions for: in accordance with a determinationthat the distance criteria is satisfied and the virtual objectcorresponds to the second real object, forgoing removing the portion ofthe displayed virtual object.
 20. The non-transitory computer-readablestorage medium of claim 18, wherein the virtual object does notcorrespond to the second real object when the virtual object is not arepresentation of the second real object in the computer-generatedreality environment.
 21. The non-transitory computer-readable storagemedium of claim 18, the one or more programs further includinginstructions for: determining whether the virtual object corresponds tothe second real object based on reconstruction data extracted from animage of a real environment captured using the one or more sensors. 22.The non-transitory computer-readable storage medium of claim 18, the oneor more programs further including instructions for: in response toremoving the portion of the displayed virtual object, presenting, in theremoved portion, a second virtual object different from the virtualobject.
 23. A method, comprising: at an electronic device: presenting acomputer-generated reality environment comprising a representation of afirst real object and a displayed virtual object; detecting, using oneor more sensors, a movement of the first real object; presenting acorresponding movement of the representation of the first real objectfrom a first location to a second location in the computer-generatedreality environment; and in accordance with a determination that adistance criteria is satisfied and the virtual object does notcorrespond to a second real object detected using the one or moresensors, removing a portion of the displayed virtual object, wherein thedistance criteria comprises a threshold distance between the secondlocation and the virtual object.
 24. The method of claim 23, furthercomprising: in accordance with a determination that the distancecriteria is satisfied and the virtual object corresponds to the secondreal object, forgoing removing the portion of the displayed virtualobject.
 25. The method of claim 23, wherein the virtual object does notcorrespond to the second real object when the virtual object is not arepresentation of the second real object in the computer-generatedreality environment.
 26. The method of claim 23, further comprising:determining whether the virtual object corresponds to the second realobject based on reconstruction data extracted from an image of a realenvironment captured using the one or more sensors.
 27. The method ofclaim 23, further comprising: in response to removing the portion of thedisplayed virtual object, presenting, in the removed portion, a secondvirtual object different from the virtual object.